Video insertion information insertion in a compressed bitstream

ABSTRACT

Particular embodiments generally relate to video insertion information insertion in a compressed bitstream. In one embodiment, a compressed bitstream of data is received. Video insertion information to be added to the bitstream is then determined. One or more macroblocks out of a plurality of macroblocks are determined in the compressed bitstream. The macroblocks are determined in the compressed domain and not the spatial domain. Information is selectively inserted for the video insertion information for the determined one or more macroblocks in the compressed bitstream. For example, encoded macroblocks for the video insertion information may replace the determined one or more macroblocks. Also, the one or more macroblocks may be decoded and information for the video insertion information is added to the decoded one or more macroblocks, and then the one or more macroblocks are re-encoded with the information. The re-encoded macroblocks may then replace the determined one or more macroblocks in the compressed bitstream.

TECHNICAL FIELD

Particular embodiments generally relate to video processing.

BACKGROUND

As video providers deliver video to consumers via telecommunication,cable, mobile, etc., revenue generated from advertising becomesincreasingly more important to their profitability and viability.Advertisements may be inserted in video that is sent to customers.Delivering advertisements that are targeted to a customer increases thevalue of advertising. For addressable advertising, an advertisement maybe inserted into video being sent to a specific customer that targetsthe customer's preferences. Advertising could also be targeted to asmall set of subscribers based on demographics i.e. locality, age,income, etc.

The video stream is typically decoded to insert video insertioninformation advertisement in it. The advertisement may then be insertedinto the decoded video, and then the video with the embeddedadvertisement is re-encoded. This adds complexity and cost toadvertising insertion systems such as splicers, advertising servers androuting systems that are now required to decode and re-encode the wholeframe for a picture with the advertisement. This increases processingrequirements and also reduces the picture quality due to the extradecode/encode cycle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an example of an end-to-end video delivery system andshows the various places video insertion information insertion can beaccomplished according to one embodiment.

FIG. 2 depicts a more detailed example of video insertion informationinserter according to one embodiment.

FIG. 3 depicts a more detailed example of the video insertioninformation inserter according to one embodiment.

FIG. 4 depicts a simplified flowchart of a method for performinginsertion of video insertion information according to one embodiment.

DESCRIPTION OF EXAMPLE EMBODIMENTS Overview

Particular embodiments generally relate to video insertion informationinsertion in a compressed bitstream. In one embodiment, a compressedbitstream of data is received. Video insertion information to be addedto the bitstream is then determined. One or more macroblocks out of aplurality of macroblocks are determined in the compressed bitstream. Themacroblocks are determined in the compressed domain and not the spatialdomain. Information is selectively inserted for the video insertioninformation for the determined one or more macroblocks in the compressedbitstream. For example, encoded macroblocks for the video insertioninformation may replace the determined one or more macroblocks. Also,the one or more macroblocks may be decoded and information for the videoinsertion information is added to the decoded one or more macroblocks,and then the one or more macroblocks are re-encoded with theinformation. The re-encoded macroblocks may then replace the determinedone or more macroblocks in the compressed bitstream.

Example Embodiments

FIG. 1 depicts an example of a system according to one embodiment.System 100 is used to deliver video to users. A simplified system 100 isshown, but it will be understood that a person skilled in the art willappreciate other elements that may be included in system 100.

A video head-end 104, a local head-end 106, and a set-top box 108 areprovided in a video network. Also, video insertion information inserter102 may be included in any location. For example, video insertioninformation may be inserted at the local head-end, set-top box, or videohead-end. Also, routers that are delivering the video content in thenetwork may also insert the video insertion information.

Video streams are typically received at Video Headend 104 via adistribution network such as satellite. The video Service Provider'sVideo head-end 104 may then re-purpose the video streams prior to bedelivering them to the user.

Local head-end 106 may be found in a central office for atelecommunications provider or any other regional office. Localhead-ends allow the Service Provider to re-groom and do localadvertisement insertion to target a finer granularity of subscriberdemographics. For example, local head-end 106 may be an office that iscloser to a user. Video head-end 104 and local head-end 106 may includevideo content delivery devices, such as routers, switches along withadvertising insertion equipment, that deliver video to set-top box 108,which may be found in a home, office, etc. Set-top box 108 may be anycomputing device that may be able to decode the video and display it ona display device 110.

Video insertion information inserter 102 is configured to insert videoinsertion information into the compressed video bitstream. For example,macroblocks may be inserted into the compressed bitstream to replaceother macroblocks. The replacement is performed when the bitstream iscompressed and not in the spatial domain, hence video insertioninformation inserter in the set-top box 108 would sit before the decoderfunctionality. The spatial domain is where the information is notencoded.

Video insertion information may be related to advertisements that areembedded in the video stream from the Video Head-End and the videoinsertion information that is added will allow for customization and/orlocalization. The video insertion information may be added in a videostream after it has been sent from video head-end 104. Video insertioninformation may be any information that is inserted into the compressedbitstream. The video insertion information may replace video in apicture completely (also called a replacement advertisement message) ormay be overlaid over the video (transparent advertisement message alsoknown as an overlay). For example, if the picture includes an image of acar, a car name may be inserted into the video bitstream. The car namemay be overlaid over the image of the car such that the car can still beseen under the car name (i.e., the car name is combined with the imageof the car). Also, the car name may replace all information in thebitstream such that a macroblock including the car name replaceswhatever information was included in a previous macroblock for the carimage.

Video insertion information may include static or dynamic information.For example, static information may include a logo or other informationthat does not change. Dynamic video insertion information may includetickers, crawls, video, or other information that changes over time.

The video insertion information is inserted by determining the numberand location of macroblocks affected by the insertion of the videoinsertion information. For example, macroblocks are determined that maybe modified by the video insertion information and are replaced in thecompressed domain. The macroblocks are replaced between referenceframes. Since only certain macroblocks are replaced, an entire framedoes not have to be processed (i.e., decoded), and only adjacentmacroblocks may be affected. This decreases the processing complexityneeded to insert the video insertion information and hence the costassociated.

FIG. 2 depicts a more detailed example of video insertion informationinserter 102 according to one embodiment. As shown, a macroblockestimator 202, a macroblock inserter 204, and a frame transmitter 206are provided. Macroblock estimator 202 receives a compressed bitstreamof data. For example, the bitstream of data include a number of frames.A frame of data may include multiple macroblocks. A stream of dataincludes multiple compressed frames or pictures. Multiple pictures forma GOP or Group of Pictures. Each GOP begins with a reference frame. Eachpicture includes slices and each slice contains a number of macroblocks.A macroblock may represent a block of data, which may be variable insize, such as 16×16, 8×16, etc. pixels.

Macroblock estimator 202 determines whether one or more macroblocks thatshould be replaced to fit the video insertion information into the videoand also which ones. Macroblock estimator 202 may use informationassociated with the video bitstream to determine which macroblocks toreplace. For example, the bitrate, macroblock size, metadata, or otherinformation may also be used to determine which macroblocks may bereplaced. In one example, a position to insert the video insertioninformation in a picture may be specified and the macroblocks includinginformation for the position are determined. Also, macroblock estimator202 may dynamically determine an optimal location in the video forinserting the video insertion information. Optimal macroblocks may bedetermined based on a number of criteria related to the bitstream, suchas a minimized motion vector, low coding density, backgroundcharacteristics, etc. For example, an area that includes very littlemotion may be a good candidate for inserting video insertioninformation. Macroblock estimator 202 is ‘aware’ of the video insertioninformation details such as size among others, that is to be insertedinto the compressed video stream.

Another embodiment is to have the compressed video insertion informationdelivered to local head-end 106 prior to being required along with aschedule for insertion into the bitstream. Metadata describing the videoinsertion information attributes such as the time of play, affectedmacroblocks, bit rate, etc., can be associated with that pre-compressedvideo insertion information and passed to macroblock estimator 202. Theinsertion of video insertion information can be triggered either by timeor by an in-band trigger mechanism. The delivery of the compressed videoinsertion information could be included as a separate PID (program ID)in the same stream.

Another embodiment is to have the compressed video insertion informationdelivered from video head-end 104 as a separate stream along with theassociated metadata that describes the video insertion informationattributes and trigger time.

Macroblock inserter 204 is configured to insert the video insertioninformation into the compressed bitstream. As will be discussed infurther detail below, the information for the video insertioninformation may completely replace the macroblocks or be overlaid(combined with information in the determined macroblocks).

Once the macroblocks have been inserted into the compressed bitstream, aframe transmitter 206 may transmit the frame. The frame transmittedincludes the video insertion information that has been inserted in theframe.

FIG. 2 also depicts an example of a frame of data 208 as it progressesthrough the processing according to one embodiment. A macroblockestimator 202 may determine macroblocks 210-1 and 210-2 should bereplaced to include the video insertion information. These macroblocksare determined.

Macroblock inserter 204 then replaces macroblocks 210-1 and 210-2 withreplacement macroblocks 212-1 and 212-2. These macroblocks are for thevideo insertion information. This is performed in the compressedbitstream.

Frame transmitter 208 then transmits the frame with replacementmacroblocks 212-1 and 212-2. As such, a decoding of frames andre-encoding is not necessary to insert the video insertion information.

The processing for when the video insertion information replaces thevideo or is overlaid in the video will now be described. FIG. 3 depictsa more detailed example of video insertion information inserter 102according to one embodiment. In one example, macroblocks 210-1 and 210-2may be completely replaced with macroblocks including information forthe video insertion information. A compressed bitstream may be receivedat macroblock inserter 204. Macroblock estimator 202 may determine whichmacroblocks should be replaced.

Before inserting the video insertion information in the compressedbitstream, the information for the video insertion information iscompressed. Video insertion information may be received at an encoder302. Encoder 302 may then encode the video insertion information intoone or more macroblocks. In one example, the encoded macroblocks for thevideo insertion information may match the number of the macroblocksbeing replaced. For example, the number of the macroblocks to bereplaced is known and may substantially match the number of themacroblocks for the video insertion information. Thus, the adjustmentsto the bitstream may be minimized because the same number of macroblockis being inserted as the macroblock being replaced. Macroblock inserter204 then replaces the one or more determined macroblocks from thecompressed bitstream with the encoded video insertion informationmacroblocks. Other embodiments would allow the macroblocks to bereplaced to be specified either in-band or out-of-band as attachedmetadata.

To replace the macroblocks, macroblock estimator 202 may determine apoint in the bitstream that corresponds to the determined macroblock. Inone embodiment, a macroblock is determined by determining the packetizedelementary stream (PES) packet boundary. The PES packet header is foundand used to determine the packet boundary. A picture header is thendetermined to find the picture boundary. The picture header contains thetype of picture that is present, i.e., I, P, B picture including atemporal reference indicating the position of the picture in displayorder within the Group of Pictures (GOP). A slice header is then foundand used to determine the slice boundary. A Slice is a string ofmacroblocks of arbitrary length running from left to right and top tobottom across the picture. The slice header contains the verticalposition of the slice within the picture. The macroblock boundary isthen determined using the macroblock location header. The macroblockheader contains the horizontal location of each macroblock within theslice, with the first macroblock header location being absolute and theothers in the slice being differential to the most recently transmittedMacroblock. The encoded macroblock may be inserted into the bitstream atthe boundary point to replace the macroblock. Macroblock estimator 202may communicate macroblock or macroblocks to be replaced location(s) inthe bitstream to video insertion information inserter 204.

The above replacement description may be for one frame; however, thevideo insertion information may be displayed for multiple frames orpictures. For example, the video insertion information may be displayedfor five seconds in the video. Thus, the video insertion informationneeds to be replaced in multiple frames. In one embodiment, macroblockinserter 204 may identify a reference I-frame and buffer it. Theoriginal macroblock is replaced with the encoded video insertioninformation macroblock starting with the Reference I-frame selected.Macroblocks are continually replaced in subsequent frames (such asB-frames, P-frames, or I-frames) with the encoded video insertioninformation for the selected time. For example, if the video insertioninformation should be visible for five seconds, the same macroblocks inmultiple frames may be replaced for frames including informationdisplayed for the five second period. The end of the video insertioninformation may then be closed with a Reference I-frame.

Macroblock inserter 204 then transmits the frame to frame transmitter206. Frame transmitter 206 may perform a bit rate adjustment or bufferfullness adjustment (vertical buffer verifier). The bit rate adjustmentmay be performed because the video characteristics associated with theoriginal bit-stream have now been modified by the video insertioninformation. Also, buffer fullness adjustment may be performed. Bit rate(from frame transmitter 206) may be adjusted to correspond to theincoming bit rate or the streaming bit rate associated with the videostream. Buffer fullness implies compliance with MPEG2 vbv (videobuffering verifier) which is adjusted per spec to eliminate decodervideo buffer overflow and underflow conditions.

In another embodiment, the information for the video insertioninformation may be combined with information in the determinedmacroblocks. In one embodiment, the information may be combined in thespatial domain and not the compressed domain. A macroblock decoder 304may decode the determined macroblocks that have been identified forreplacement. The decoded macroblocks may be combined with the videoinsertion information in the spatial domain. The combination isperformed only for the macroblocks affected by the video insertioninformation in one embodiment, and not for macroblocks that are notaffected. The not affected macroblocks may not be decoded, which reducesprocessing overhead for decoding/encoding the whole frame.

An encoder 306 encodes the combined macroblocks. Then, macroblockinserter 204 may replace the determined macroblocks with the macroblocksthat include the combined information. Although the macroblocks aredecoded and combined with information for the video insertioninformation, the replacement is performed in the compressed bitstream.For example, the new macroblocks replace the previous macroblocks in thecompressed bitstream.

FIG. 4 depicts a simplified flowchart 400 of a method for performinginsertion of video insertion information according to one embodiment.Step 402 receives a compressed bitstream of data. Step 404 determinesvideo insertion information to be inserted into the bitstream. Step 406determines one or more macroblocks in the bitstream that should bereplaced.

The replacement may then be a complete replacement or video insertioninformation. Step 408 determines if it is a complete replacement. Ifnot, step 410 decodes the determined one or more macroblocks. Step 412combines information for the video insertion information with thedecoded one or more macroblocks. Step 414 then re-encodes themacroblocks. Step 416 replaces the one or more macroblocks with themacroblocks that include the combined information.

If the macroblocks are to be replaced, step 418 replaces the determinedmacroblocks with the macroblocks for the video insertion information.

Particular embodiments provide many advantages. Minimized processing isrequired because only macroblocks affected by the video insertioninformation may be processed. Also, reduced cost because the generatorof the video insertion information is not required at local head-end106. Rather, the video insertion information may be generated prior tothe insertion and distributed to local head-end sites prior to orsimultaneous with the content being delivered.

Although the description has been described with respect to particularembodiments thereof, these particular embodiments are merelyillustrative, and not restrictive. Although advertisements aredescribed, it will be understood that any information may be inserted ina video stream.

Any suitable programming language can be used to implement the routinesof particular embodiments including C, C++, Java, assembly language,etc. Different programming techniques can be employed such as proceduralor object oriented. The routines can execute on a single processingdevice or multiple processors. Although the steps, operations, orcomputations may be presented in a specific order, this order may bechanged in different particular embodiments. In some particularembodiments, multiple steps shown as sequential in this specificationcan be performed at the same time.

A “computer-readable medium” for purposes of particular embodiments maybe any medium that can contain, store, communicate, propagate, ortransport the program for use by or in connection with the instructionexecution system, apparatus, system, or device. The computer readablemedium can be, by way of example only but not by limitation, anelectronic, magnetic, optical, electromagnetic, infrared, orsemiconductor system, apparatus, system, device, propagation medium, orcomputer memory. Particular embodiments can be implemented in the formof control logic in software or hardware or a combination of both. Thecontrol logic, when executed by one or more processors, may be operableto perform that which is described in particular embodiments.

Particular embodiments may be implemented by using a programmed generalpurpose digital computer, by using application specific integratedcircuits, programmable logic devices, field programmable gate arrays,optical, chemical, biological, quantum or nanoengineered systems,components and mechanisms may be used. In general, the functions ofparticular embodiments can be achieved by any means as is known in theart. Distributed, networked systems, components, and/or circuits can beused. Communication, or transfer, of data may be wired, wireless, or byany other means.

It will also be appreciated that one or more of the elements depicted inthe drawings/figures can also be implemented in a more separated orintegrated manner, or even removed or rendered as inoperable in certaincases, as is useful in accordance with a particular application. It isalso within the spirit and scope to implement a program or code that canbe stored in a machine-readable medium to permit a computer to performany of the methods described above.

As used in the description herein and throughout the claims that follow,“a”, “an”, and “the” includes plural references unless the contextclearly dictates otherwise. Also, as used in the description herein andthroughout the claims that follow, the meaning of “in” includes “in” and“on” unless the context clearly dictates otherwise.

Thus, while particular embodiments have been described herein, alatitude of modification, various changes and substitutions are intendedin the foregoing disclosures, and it will be appreciated that in someinstances some features of particular embodiments will be employedwithout a corresponding use of other features without departing from thescope and spirit as set forth. Therefore, many modifications may be madeto adapt a particular situation or material to the essential scope andspirit.

1. A method comprising: receiving a compressed bitstream of data;determining video insertion information to be added to the bitstream;determining one or more macroblocks out of a plurality of macroblocks inthe compressed bitstream; and selectively inserting information in thevideo insertion information for the determined one or more macroblocksin the compressed bitstream.
 2. The method of claim 1, whereinselectively inserting comprises replacing the determined one or moremacroblocks in the compressed bitstream with one or more macroblocksincluding the information for the video insertion information.
 3. Themethod of claim 1, further comprising: analyzing the bitstream; anddetermining an optimal location to insert the video insertioninformation based on one or more criteria associated with the compressedbitstream.
 4. The method of claim 1, wherein selectively insertingcomprises: decoding the determined one or more macroblocks; adding thevideo insertion information into the decoded determined one or moremacroblocks; encoding the one or more macroblocks with the added videoinsertion information; and inserting the encoded one or more macroblocksfor the determined one or more macroblocks in the compressed bitstream.5. The method of claim 1, wherein determining the one or moremacroblocks comprises determining a number of macroblocks that need tobe replaced to fit the information in the overlap in the compressedbitstream.
 6. The method of claim 1, wherein the information for thevideo insertion information comprises one or more macroblocks that are asame number as the determined one or more macroblocks.
 7. The method ofclaim 1, wherein determining the one or more macroblocks comprises:determining a location to insert the video insertion information intothe compressed bitstream based on one or more criteria; and determiningone or more macroblocks in the location.
 8. The method of claim 1,further comprising: determining a time period for displaying the videoinsertion information; and determining the one or more macroblocks basedon the time period.
 9. The method of claim 1, wherein insertingcomprises: starting the insertion at a first I frame; continuing theinserting in subsequent frames; and ending the insertion at a second Iframe.
 10. An apparatus comprising: one or more processors; and logicencoded in one or more tangible media for execution by the one or moreprocessors and when executed operable to: receive a compressed bitstreamof data; determine video insertion information to be added to thebitstream; determine one or more macroblocks out of a plurality ofmacroblocks in the compressed bitstream; and selectively insertinformation in the video insertion information for the determined one ormore macroblocks in the compressed bitstream.
 11. The apparatus of claim11, wherein logic operable to selectively insert comprises logicoperable to replace the determined one or more macroblocks in thecompressed bitstream with one or more macroblocks including theinformation for the video insertion information.
 12. The apparatus ofclaim 11, wherein the logic when executed is further operable to:analyze the bitstream; and determine an optimal location to insert thevideo insertion information based on one or more criteria associatedwith the compressed bitstream.
 13. The apparatus of claim 11, whereinlogic operable to selectively insert comprises logic further operableto: decode the determined one or more macroblocks; add the videoinsertion information into the decoded determined one or moremacroblocks; encode the one or more macroblocks with the added videoinsertion information; and insert the encoded one or more macroblocksfor the determined one or more macroblocks in the compressed bitstream.14. The apparatus of claim 11, wherein logic operable to determine theone or more macroblocks comprises logic is further operable to determinea number of macroblocks that need to be replaced to fit the informationin the overlap in the compressed bitstream.
 15. The apparatus of claim11, wherein the information for the video insertion informationcomprises one or more macroblocks that are a same number as thedetermined one or more macroblocks.
 16. The apparatus of claim 11,wherein logic operable to determine the one or more macroblockscomprises logic is further operable to: determine a location to insertthe video insertion information into the compressed bitstream based onone or more criteria; and determine one or more macroblocks in thelocation.
 17. The apparatus of claim 11, wherein the logic when executedis further operable to: determine a time period for displaying the videoinsertion information; and determining the one or more macroblocks basedon the time period.
 18. The apparatus of claim 11, wherein logicoperable to insert comprises logic further operable to: start theinsertion at a first I frame; continue the inserting in subsequentframes; and end the insertion at a second I frame.
 19. An apparatuscomprising: means for receiving a compressed bitstream of data; meansfor determining video insertion information to be added to thebitstream; means for determining one or more macroblocks out of aplurality of macroblocks in the compressed bitstream; and means forselectively inserting information in the video insertion information forthe determined one or more macroblocks in the compressed bitstream.